# Self-Distillation Training

Introducing an attention gating (AttnGates) weight adapter for the QwQ-32B model to accelerate long-context computation through dynamic block-level sparsity

SPLADE model for passage retrieval, improving retrieval effectiveness through sparse latent document expansion and knowledge distillation techniques

An unsupervised sentence encoder based on RoBERTa-large, trained with self-distillation and mutual distillation techniques, suitable for sentence similarity calculation tasks.